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DETAILED ACTION 

Response to Amendment 

1 . In response to the office action from 2/1 5/2005, the applicant has submitted an 
amendment, filed 6/15/2005, arguing to traverse the art rejection based on the limitation 
regarding the rebuilding of a speech model for each of a pre-selected number of linguistic 
features (Amendment, Pages 8-9). Applicant's arguments have been fully considered, however 
the previous rejection is maintained due to the reasons listed below in the response to arguments. 

Response to Arguments 

2. Applicant's arguments have been fully considered but they are not persuasive for the 
following reasons: 

With respect to Claims 1, 13, and 25, the applicant argues that De Souza et al (U.S. 
Patent: 5,884,261) fails to teach rebuilding a speech model for ranked linguistic features 
(Amendment, Page 9), however the examiner notes that it is the combination of Nouza ("Feature 
Selection Methods for Hidden Markov Model-based Speech Recognition"), Eide et al ("A 
Linguistic Feature Representation of the speech Waveform," 1993) and De Souza that teaches 
this feature. 

Nouza provides the teachings of obtaining speech data and building a model for each for 
feature of an original set of speech features (Page 187, Col 1, Lines 5-9; Page 188, Col 1, Lines 
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5-1 3; and Prior office action, Page 3). Nouza also teaches a means for ranking speech features 
(Page 188, Col. 1, last paragraph- Col. 2, first paragraph). Nouza is deficient in the specific 
teachings of the use of linguistic features in describing speech and rebuilding a speech model for 
a number of ranked linguistic features. The inclusion of Eide provides the teaching of the use of 
linguistic speech features (Pages 483-484, Section 1; and Table 1) in place of the speech features 
disclosed by Nouza for the benefit of improving speech recognition accuracy through contextual 
information provided by linguistic features (Page 485-486, Section 3; and Table 5). 

Nouza in view of Eide is deficient in the specific teachings of rebuilding a speech model 
for ranked features, however the examiner points out that Nouza does note the importance of 
reducing the size of an original HMM model by utilizing ranked features, which would require a 
necessary step of original model rebuilding for the ranked features (Nouza, HMM simplification, 
Page 190, Section 7). Additionally, De Souza specifically teaches rebuilding a Markov model 
by replacing original model features with features that have a highest likelihood of an acoustic 
match (Col. 16, Lines 15-59) for the benefit of creating updated models which best represent 
speech feature data (Col. 16, Lines 38-42). The process of replacing portions of the Markov 
speech model taught by De Souza is the equivalent of rebuilding a speech model for ranked 
features in the present invention, since the determination of the highest likelihood of an acoustic 
match in De Souza would require feature ranking (which is also taught by Nouza as noted 
above). Eide discloses the specific use of linguistic features, and when taken in combination 
with Nouza and De Souza, teaches the limitation regarding the rebuilding of a speech model for 
ranked linguistic features. 
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The dependent claims are argued as further limiting rejected independent claims 
(Amendment, Page 10), and thus, also remain rejected. 

Also, the applicant has not officially challenged the official notice taken with respect to 
claim 25 regarding the use of a computer readable medium in any of the prior office actions, 
thereby making the use of such a medium the applicant's admitted prior art. 

Claim Rejections - 35 USC §103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

4. Claims 1- 24 are rejected under 35 U.S.C. 103(a) as being unpatentable over Nouza 

( "Feature Selection Methods for Hidden Markov Model-based Speech Recognition ") in view of 
Eide et al ("A Linguistic Feature Representation of the Speech Waveform, " 1993), and further in 
view of De Souza et al (U.S. Patent: 5,884,261). 

With respect to Claims 1 and 13, Nouza recites: 

Obtaining speech input data (HMM and DTW speech recognition systems, Page 188, Col. 
1, Lines 5-7; Inherently, speech data would have to be received in order for speech to be 
recognized by the recognition system.)', 

Building a model for each feature of an original set of features (parameters used to 
distinguish models of different speech objects in the form of Gaussian mixture pdfs, Page 187, 
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Col. 1, Lines 5-9, and evaluated for individual feature contributions for speech unit 
classification, Page 188, Col 1, Lines 11-13); 

Ranking the features (feature significance factor that can be used for ordering features, 
Page 188, Col. 1, last paragraph - Col. 2, first paragraph); 

Nouza does not teach the use of linguistic features in building speech models for 
recognition, however, Eide discloses a method for creating speech recognition models using 
speech features that have been linguistically classified (Pages 483-484, Section 1, and Table 1). 

Nouza and Eide are analogous art because they are from a similar field of endeavor in 
speech recognition feature processing. Thus, it would have been obvious to a person of ordinary 
skill in the art, at the time of invention, to combine the use of linguistic features in the creation of 
a speech model for recognition as taught by Eide with the speech recognition system utilizing 
feature selection as taught by Nouza to improve recognition accuracy through contextual 
information provided by linguistic features, thus implementing a means of keyword spotting 
(Eide, Page 485-486, Section 3 and Table 5). 

Neither Nouza nor Eide explicitly teach the additional step of rebuilding the model for 
each of a preselected number of ranked features, however De Souza discloses a means for 
updating speech recognition model arcs utilizing speech features having a highest likelihood of 
an acoustic match (Col 16, Lines 15-59). 

Nouza, Eide, and De Souza are analogous art because they are from a similar field of 
endeavor in speech recognition feature processing. Thus, it would have been obvious to a person 
of ordinary skill in the art, at the time of invention, to modify the teachings of Nouza in view of 
Eide with the ability to rebuild speech recognition model arcs utilizing speech features having a 
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highest likelihood of an acoustic match as taught by De Souza in order to implement more 
accurate speech recognition by creating updated models which best represent speech feature data 
(De Souza, Col 16, Lines 38-42). 

With respect to Claims 2 and 14, Nouza further discloses: 

The method and apparatus according to claims 1 and 13, respectively, wherein said step 
of building a model for each of a pre-selected number N of the ranked features comprises 
building a model for the top N ranked features (reducing the size of feature vectors used in 
speech processing, Abstract, Lines 5-6, feature significance factor that can be used for ordering 
features, Page 188, Col. 1, last paragraph - Col 2, first paragraph, and identifying correct and 
incorrect speech models based upon those features, Page 188, Lines 26-32. It would be 
inherent, upon selection of principal components with the largest amount of variance from an 
ordered component set used for separating hypothesis choices, that the principal component 
features being of a highest likelihood to represent a particular state within a HMMbe used to 
remodel a most likely HMM candidate for speech recognition.) 

With respect to Claims 3 and 15, Nouza additionally recites: 

The method and apparatus according to claims 1 and 13, respectively, further comprising 
the step of compiling a confusion matrix for each feature of the original set of features 
subsequent to said step of building a model for each feature of an original set of features 
(covariance matrix used to evaluate the contributions of a feature in speech classification and to 
determine whether a particular speech model is correct or incorrect, Page 188, Col 1, Lines 11- 
32). 

With respect to Claims 4, 5, 16, and 17, Nouza further discloses: 
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The method and apparatus according to claims 3 and 15 and claims 4 and 16, 
respectively, wherein said step of compiling a confizsion matrix comprises computing a score for 
each feature based on the likelihood, as a log-likelihood as per Claim 5, of its presence in a frame 
of the speech input data (contribution of a feature within a covariance matrix in identifying a 
speech unit using a particular speech model, which is represented by a log-likelihood score, 
Page 188, Col 1, Lines 13-23). 

With respect to Claim 6 and 18, Nouza teaches the method and system of feature 
selection in recognizing a speech unit, utilizing a confusion matrix used to evaluate the 
contributions of a feature in speech classification and to determine whether a particular speech 
model is correct or incorrect as applied to Claims 3 and 15, while De Souza teaches the means of 
rebuilding speech recognition model arcs utilizing speech features having a highest likelihood of 
an acoustic match, as applied to Claims 1 and 13. Neither Nouza nor De Souza specifically 
teaches comparing likelihood scores to a predetermined threshold as a means of detecting 
whether a speech feature is useful in picking a correct classification however, Eide discloses: 

Compiling a confusion matrix further comprises comparing each score of each feature 
with a threshold (detection of a particular linguistic feature within a phoneme that would 
inherently require some type of threshold comparison to determine the presence of such a 
feature, Pages 484-484, Section 1 and Tables 1-4). 

Nouza, Eide, and De Souza are analogous art because they are from a similar field of 
endeavor in speech recognition feature processing. Thus, it would have been obvious to a person 
of ordinary skill in the art, at the time of invention, to modify the teachings of Nouza and De 
Souza with the use of a threshold comparison in determining the presence of a particular 
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linguistic feature within a phoneme as suggested by Eide to provide a well-known and 
convenient means of detecting if a linguistic feature is present in picking a correct phoneme 
classification through threshold comparison. Therefore, it would have been obvious to combine 
Eide, Nouza, and De Souza for the benefit of detecting the presence of a particular linguistic 
feature for phoneme classification. 

With respect to Claims 7 and 19, Eide additionally discloses: 

Calculating mutual information between truth and labels for each feature (determination 

of the absence or presence of a particular speech feature designated by a "+ " or " in 

phoneme classification, Pages 483-484, Section 1, and Tables 1-4), 
With respect to Claims 8 and 20, Eide further recites: 
Ranking the mutual information calculated in compiling the confusion matrix 

(determination of the most-likely linguistic classes used to describe a phoneme, which would 

inherently require a step of linguistic feature ranking, Pages 483-484, and Tables 1-4). 
With respect to Claims 9, 11, 21, and 23, Eide additionally recites: 
Partitioning the speech input data in parallel, once fors each linguistic feature (dividing 

speech training data according to linguistic feature truth labels, Page 483, Section 1)\ and 
Producing an observation vector (calculating attribute vectors, Page 483, Section 1), 
With respect to Claims 10, 12, 22, and 24, Eide further discloses: 
Portioning data in parallel from the observation vector, once for each feature (dividing 

attribute vectors into feature-present and feature-absent sets for all linguistic features, Page 484, 

Section 1); and 
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Producing final observations (final determination of whether a particular linguistic 
feature is present or absent in speech training data, Page 484, Section 1). 

5. Claim 25 is rejected under 35 U.S.C. 103(a) as being unpatentable over Nouza {"Feature 
Selection Methods for Hidden Markov Model-based Speech Recognition ") in view of Eide et al 
("A Linguistic Feature Representation of the Speech Waveform, " 1993), further in view of De 
Souza et al (U.S. Patent: 5,884,261), and yet further in view of the applicant's admitted prior 
art. 

With respect to Claim 25, Nouza in view of Eide, and in further view of De Souza 
teaches the method of linguistic feature selection in building a speech recognition unit as applied 
to Claim 1. While Nouza in view of Eide, and in further view of De Souza, does not teach it, it is 
the applicant's admitted prior art to implement the method taught by Nouza in view of Eide 
using a computer program contained on a computer storage device, since computers are 
conveniently used and their programs easily updated for performing speech recognition 
operations, while a storage device would offer a means of storing any training databases or other 
necessary stored information. Therefore, it would have been obvious to one of ordinary skill in 
the art, at the time of invention, to implement a linguistic feature selection method for 
recognition model building using a computer program transferable between various machines 
through the use of a storage device, thus increasing method adaptability, to obtain the invention 
as specified in Claim 25. 
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Conclusion 

6. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing ' 
date of this final action. 

7. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure: 

Jiang et al (U.S. Patent: 6,542,866)- teaches a means for ranking speech features for the 
determination which features best represent speech, which utilizes acoustic models for each 
linguistic unit of a language. 

8. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to James S. Wozniak whose telephone number is (571) 272-7632 
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and email is James.Wozniak@uspto.gov. The examiner can normally be reached on Mondays- 
Fridays, 8:30-4:30. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Wayne Young can be reached at (571) 272-7582. The fax/phone number for the 
Technology Center 2600 where this application is assigned is (703) 872-9306. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the technology center receptionist whose telephone number is (703) 306- 
0377. 

James S. Wozniak 
7/22/2005 



PRIMARY EXAMINER 




